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(54) - Compressing digital Images 



(57) A method for producing a compressed digitai 
image from an input digital image is disclosed, wherein 
the compressed digital image Is organized Into layers 
having fnfomriation that can be extracted In accordance 
with different desired viewing conditions and resolu- 
tions. The input digital image is decomposed to produce 
a plurality of subbands, each subband having a plurality 
of subband coefficients. The plurality of subband coef- 
ficients of each subband of the decomposed input digital 
Image are quantized to produce a quantized output val- 
ue for each subband coefRcient of each subt>and. At ' 
least one bit-plane Is formed from the quantized output 
values of subband coefficients of each subband. Each 
bit-plane of each subband in at least one pass is entropy 
encoded to produce a compressed bit-stream corre- 
sponding to each pass, wherein each subband is entro- 
py encoded Independently of the other subbands. A vis- 
ual quality table is provided that specifies a number of 
expected visual quality levels by providing selectable 
viewing conditions and resolutions for each expected 
visual quality level. A minimal set of passes and their 
con-esponding compressed bit-streams that are neces- 
sary to satisfy the expected visual quality levels provid- 
ed in the visual quality table are identified, whereby a 
user can select different desired viewing conditions and 
resolutions for each compressed Image. The com- 
pressed bit-streams corresponding to passes are then 
ordered Into layers from the lowest expected visual qual- 
ity level to the highest expected visual quality level spec- 
ified in tiie visual quality table to produce a compressed 



digital image, Vk^erein each layer includes the passes 
and their corresponding compressed bit-streams from 
the identified minimal set corresponding to the expected 
visual quality level that have not been Included In any 
lower visual quality layers. 
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Description 

[0001] This invention describes a method for produc- 
ing a compressed digital image organized into layers 
having Information relating to different viewing condi- 
tions and resolutions. 

[0002] Subband or wavelet bit-plane coding is being 
used in the proposed J PEG2000 standard, as described 
in ISO/lEC JTC1/SC29 WG1 N1646, JPEG2000 Part I 
Final Committee Draft, Version 1.0, March 2000. The 
JPEG2000 encoder decomposes the image into a hier- 
archy of resolutions and the compressed data corre- 
sponding to a resolution is further divided into a number 
of quality layers. At any resolution, adding more layers 
to the compressed bit-stream generally improves the 
visual quality of the image reconstructed at that resolu- 
tion and at higher resolutions. The JPEG2000 standard 
provides a very flexible frameworic for organizing and 
ordering the compressed bit-stream. For example, a 
compressed bit-stream may be resolution-layer-compo- 
nent-position progressive or layer-resolution-compo- 
rwnt-positlon progresshfe. Such bit-streams are also re- 
ferred to as resolution-scalable and quality-scalable, re- 
spectively. It is up to the individual JPEG2000 encoder 
to an^nge the compressed bit-stream in a manner suit- 
able for the intended application. 
[0003] if the compressed bit-stream is arranged In a 
resolution-layer-component-position progressive man- 
ner, henceforth referred to as resolution-progressive, all 
the layere corresponding to a lower resolution image ap- 
pear in the bit-stream eariler than any layers corre- 
sponding to higher resolutions. On the otiier hand, if the 
compressed bit-stream is arranged in a layer-resoiution- 
component-posltion progressh^e manner, henceforth re- 
ferred to as layer-progressive, iayere with lower Indices 
from all resolution levels appear in the bit-stream eariier 
than any layers with higher indices. Thus, it is not pos- 
sible for the compressed bit-stream to be resolution-pro- 
gressive and layer-progressive at the same time. But in 
some applications, a comt>ination of these two modes 
may fc>e desirable. 

[0004] For example, consider a digital camera having 
four visual quality levels. Suppose that the images are 
captured at a resolution of 1536 by 1024 pixels, but for 
the two lowest visual quality levels, the images are 
stored at the next lower resolution. 768 by 512 pixels. 
Consider a usage scenario in which the image is com- 
pressed at the highest visual quality level, but due to 
memory constraints It is necessary to truncate the com- 
pressed bit-stream so that the image is stored at the low- 
est visual quality level. If the original compressed bit- 
stream had been an^nged in a layer-progressive man- 
ner, the truncated bit-stream would still contain some 
layere from the higher (1536 by 1024) resolution. This 
Is a waste of memory space because at the lowest visual 
quality level, the Image will be stored at the 768 by 51 2 
resolution. Similariy, suppose that the compressed bit- 
stream corresponding to the highest visual quality level 



is arranged in a resolution-progressive manner, and Is 
then truncated to the next lower visual quality level at 
the same resolution, in this case, the truncated bit- 
stream may contain layere from the 768 by 512 resolu- 
5 tion that are not necessary to achieve the desired visual 
quality level. 

[0005] Also consider the situation where the digital 
camera stores many images on a flash memory card, 
and includes an image display, such as a color liquid 
10 crystal display (LCD), with a limited dynamic range. The 
LCD displays several "thumbnail size' (for example, 192 
by 128 resolution) images at the same time to allow the 
user to locate an image of interest It then displays a 
single "screennaii" size (for example 384 by 256 rese- 
ts lution) Image of one of the selected thumbnail images, 
in this situation, if the file Is arranged In a layer-pregres- 
slve manner, it will take time to sldp over some iayere 
from the higher (1536 by 1024) resolution in order to lo- 
cate Just the data needed for the lower resolution Imag- 
20 es. If the data is arranged in a resolution-progressive 
manner, it will again take time to skip over some of the 
layere from the 192 by 128 and 384 by 256 resolution 
levels that are not needed to provide an acceptable im- 
age on the low dynamic range camera LCD. As a result 
2S the time required to decode the Images will t>e slow, and 
the display will not be as responsh^e to user commands 
to display a new Image as would be desirable. 
[0006] Accordingly, an object of the present Invention 
is to provide a method for the formation of iayere of a 
30 compressed JPEG2000 bit-stream in such a manner 
that the bit-stream provides partial resolutfon scalability 
within a layer-progressive ordering. 
[0007] This object is achieved by a method for pro- 
ducing a compressed digital image from an input digital 
35 image, wherein the compressed digital image Is organ- 
ized into layere having infonmation that can be extracted 
in accordance with different desired viewing conditions 
and resolutions, comprising the steps of. 

40 (a) decomposing the input digital image to produce 
a plurality of subbands, each subband having a plu- 
rality of subband coefficients; 

(b) quantizing the plurality of subband coefficients 
of each subband of the decomposed input digital 

45 Image to produce a quantized output value for each 
subband coefficient of each subtxand; 

(c) forming at least one bit-plane from the quantized 
output values of subband coefHcients of each sub- 
band; 

so (d) entropy encoding each bit-piane of each sut>- 
band in at least one pass to produce a compressed 
bit-stream corresponding to each pass, wherein 
each subband is entropy encoded independently of 
the other subbands; 

55 (e) providing a visual quality table that specifies a 
number of expected visual quality levels by provid- 
ing selectable viewing conditions and resolutions 
for each expected visual quality level; 
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(f) identifying a minimal set of passes and their cor- 
responding compressed bit-streams that are nec- 
essary to satisfy the expected visual quality levels 
provided in the visual quality table, whereby a user 
can select different desired viewing conditions and s 
resolutions for each compressed image; and 

(g) ordering the compressed bit-streams corre- 
sponding to passes Into layers from the lowest ex- 
pected visual quality level to the highest expected 
visual quality level specified in the visual quality ta- 
ble to produce a compressed digital image, wherein 
each layer includes the passes and their corre- 
sponding compressed bit-streams from the identi- 
fied minimal set corresponding to the expected vis- 
ual quality level that have not been included In any is 
lower visual quality layers. 

[0008] The present invention provides an advantage 
in that the compressed bit-stream con^esponding to the 
highest visual quality level can be truncated to achieve 20 
any other visual quality level. 

[0009] The present invention also provides an advan- 
tage in that the truncated bit-stream contains only the 
layers that are necessary to represent the image at the 
resolution intended for that visual quality level. 2S 
[0010] In describing the preferred embodiments of the 
invention reference will be made to the series of figures 
and drawings briefly described t>etow. 

FIG. 1 shows a flow chart of an image encoder ac- ^ 
cording to the present invention; 
FIG. 2 sliows a resolution hierarchy created by a 3 
level octave subband decomposition; 
FIG. 3 shows a blocic diagram of the codeblock com- 
pression unit of FIG. 1; 35 
FIGS. 4A and 4B show graphs of the decision 
thresholds and reconstruction levels for step-sizes 
of A and 2 A, respectively, for a uniform scalar quan- 
tizer with a dead-zone; 

FIG. 5 shows a flow chart of the "layer formation 40 
and ordering decision unlT of FIG. 1; 
FIG. 6 shows a flow chart of another emt>odinnent 
of the "layer formation and ordering decision uniT 
of FIG. 1; 

FIG. 7 shows a flow chart for reconfiguring a com- 
pressed bit-stream to provide partial resolution 
scalability within a layer-progressive ordering; 
FIG. 8 shows a block diagram of a digital camera 
system including a digital camera and a digital com- 
puter which can utilize the present invention; and so 
FIG. 9 shows the back of the digital camera depict- 
ed in FIG. 8, Including ttie liquid crystal display 
(LCD). 

[0011] There may be additional structures descrit>ed ss 
in the foregoing application that are not depicted on one 
of the described drawings. In the event such a structure 
is described, but not depicted in a drawing, the absence 



of such a drawing should not be considered as an omis* 
slon of such design from the specificatton. 
[0012] The present Invention relates to compression 
of a digital image. Although there are other techniques 
well known in the art the present invention will be do- 
scribed with respect to the technique set forth in the 
JPEG2000 image compression standard. Because the 
proposed JPEG2000 image compression standard 
specifies how the decoder shall interpret a compressed 
bit-stream, there are certain inherent restrictions on any 
JPEG2000 encoder. For example. In Part I of the stand- 
ard, only certain wavelet filters can be used. The entropy 
coder is also fixed. These methods are descrit>ed in ISO/ 
lEC JTC1/SC29 WG1 N1646. JPEG2000 Part 1 Final 
Committee Draft. Version 1.0, March 2000. Hence, the 
present description will be directed in particular to at- 
tributes forming part of, or cooperating mora direcUy 
with, the algorithm in accordance witti the present in- 
vention. Attributes not specifically shown or described 
herein may be selected from those described in ISO/IEC 
JTC1/SC29 WG1 N1646. JPEG2000 Part I Final com- 
mittee Draft, Version 1.0, March 2000. or otherwise 
known in the art in the following description, a preferred 
embodiment of the present invention would ordinarily t>e 
implemented as a software program, although those 
skilled in the art will readily recognize that the equivalent 
of such software may also be constructed in hardware. 
Given the system and methodology as described in the 
following materials, all such software implementation 
needed for practice of the invention is conventional and 
within the ordinary skill in such arts. If the invention is 
implemented as a computer program, the program may 
t>e stored In conventional computer readable storage 
medium, whk^h may comprise, for example; magnetic 
storage media such as a magnetic disk (such as a floppy 
disk) or magnetic tape; optical storage media such as 
an optical disc, optical tape, or machine readat>le bar 
code; solid state electronic storage devices such as ran- 
dom access memory (RAM), or read only memory 
(ROM); or any other physical device or medium em- 
ployed to store a computer program. 
[0013] Reference will now be made in detail to the 
present preferred embodiment of the invention, an ex- 
ample of whteh Is illustrated in the accompanying draw- 
ings. While the invention will be described in connection 
with a prefen^ embodiment it will k>e understood that 
it is not intended to limit the invention to that emtxxii- 
ment On the contrary, it is intended to cover ail alterna- 
tives, modifications, and equivalents as may t>e included 
within the spirit and scope of the invention defined in the 
appended claims. 

[001 4] A flow chart of a JPEG2000 image encoder ac- 
cording to the present Invention is shown in FIG. 1. A 
digital image (101) undergoes subband decomposition 
(102) by the analysis filters to produce an image repre- 
sentation in terms of subband coefficients (103). if the 
image has multiple components (for example, RGB), a 
luminance-chrominance transformation can t>e applied 
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to convert It to a YCbCr representation, before the sut>- 
band decomposition step (102). Also, it Is possible to 
divide each component of the image into multiple tiles. 
But In this preferred embodiment only a single tile con- 
sisting of the entire image is used. The subband coeffi- s 
cients (103) are partitioned into rectangular blocks by 
the codeblock partitioning unit (104) to produce one or 
more codeblocks (105). Those skilled In the art would 
appreciate that partitioning of the subband coefficients 
is not necessary if only a single codeblock is used. Each io 
codeblock is compressed by the codeblock compres- 
sion unit (106) using the appropriate quantizer step-size 
(109) to produce a compressed codeblock (107) and a 
byte-count table (108). For each codeblock, the com- 
pressed bit-stream (107) and ttie byte-count table, de- is 
noted by TableB (108). are fed to a layer fomiation and 
ordering decision unit (111). The other Inputs to the layer 
formation and decision unit (1 11 ) are the quantizer step- 
size (109) used to quantize that codeblock and a table, 
denoted by TableQ (110), containing Information about 20 
the desired visual quality levels. Intended resolution, 
and viewing condition parameters fertile respective vis- 
ual quality levels. For each codeblock. the layer forma- 
tion and ordering decision unit (111) determines how 
many coding passes should be Included in each layer, 25 
and produces a layered compressed codeblock (112) 
and a table, denoted by Tablet (113). TableL stores in- 
formation about tiie number of coding passes and the 
corresponding bytes In each layer for that codeblock. 
The layer formation and ordering decision unit (111) also 30 
specifies that the overall bit-stream is to be arranged In 
a layer-progressive manner. The layered compressed 
codeblocks (112), TableL (113), and the ordering Infor- 
mation are fed to ttie JPEG2000 bit-stream organizer 
(114) to produce an encoded digital image (115) that is 35 
JPEG2000 compliant The master table generator (1 1 6) 
generates TableML (117) whose 1*^ entry specifies tiie 
number of bytes required to represent the compressed 
image data corresponding to the first 1 layere. This infor- 
mation is also contained in the compressed bit-stream, 40 
but in some applications It may be advantageous to 
store the information separately so that It Is not neces- 
sary to parse the bit-stream for the information. 
[0015] The blocks in FIG. 1 will now be described in 
greater detail. The image, undergoes subband deoom- 45 
position by analysis filtare (102). A single level of de- 
composition splits the image into 4 subtends, denoted 
by LL. HL. LH. and HH. The first letter Indicates the type 
of filtering employed in the horizontal direction. The sec- 
ond letter Indicates the type of filtering employed in the so 
vertical direction. For example, band HL is the result of 
high-pass filtering in the horizontal direction and low- 
pass filtering in tiie vertical direction. In an octave de- 
composition, the LL band Is split recureh^ely. In the pre- 
fen^d embodiment an R level octave decomposition is ss 
used. FIG. 2 shows an example of a 3 level octave de- 
composition. The subbands are numbered from the low- 
est frequency to tiie highest frequency. An R level de- 



composition generates (R+1) resolutions and (3R+1) 
subbands, denoted by So. S3R. An image at reso- 
lution r (0 ^ r ^ R) can be reconstructed from subbands 

So. S^ If ttie original image is of size 1024 by 

1024, and the decomposition Is 3 level octave, images 
at resolutions 0, 1 . 2 and 3 have sizes 128 by 128. 256 
by 256, 512 by 512, and 1024 by 1024, respectively. 
[0016] The codeblock partitioning unit (1 04) shown in 
FIG. 1 partitions each subband Into a number of rectarv 
gular codeblocks. The codeblock compression unit 
(106) is shown in greater detail in FIG. 3. Each code- 
block is quantized witii a scalar quantizer (201) to pro- 
duce a sign-magnitude representation of the Indices of 
quantized coefficients (202). Preferably, a unlfonm sca- 
lar quantizer with a dead-zone is used. The dedsion 
thresholds and reconstruction levels for this quantizer 
are shown in FIGS. 4A and 48. FIG. 4A shows ttie de- 
cision thresholds and reconstruction levels for a step- 
size of A; FIG. 4B shows the decision thresholds and 
reconstruction levels for a step-size of 2 A. In a preferred 
embodiment the reconstruction levels are always at the 
center of the quantization Interval. But those skilled In 
the art will recognize that this is not necessary. For ex- 
ample, the reconstruction levels can be biased towards 
zero. The same base quantizer step-size is used for all 
the codeblocks In a given subband. Let the step-size for 
subband i be A]. It should be noted ttiat the maximum 
quantization enx)r. denoted by E^ax* *s (A| / 2). except 
forthe zero bin which has a maximum quantization error 
of Af. If the subband analysis and synthesis filters are 
revereible (R. Calderbank. 1. Daubechies, W. Sweldens, 
and B.-L. Yeo. "Wavelet Transform that Maps Integere 
to integere." Applied and Computational Harmonic 
Analysis, vol. 5, no. 3. pp. 332-369, 1998), the quanti- 
zation step may be entirely absent 
[0017] Suppose that the codeblock being processed 
is from subband I. Then, the samples from tiie block are 
quantized with a uniform scalar quantizer with step size 
A| as described above. Suppose that the magnitude of 
the indices of quantized coefTidents Is represented by 
a fixed precision of A^ bits. Let the bits be Indexed as 
1.2.....Ai. where index 1 corresponds to the most signif- 
icant bit (MSB) and A{ corresponds to tiie least signifi- 
cant bit (LSB). The k**^ bit-plane for the codeblock con- 
sists of the k*^ bit from the magnitude representation of 
all the indices of quantized coefficients from that code- 
block. One Interesting property of the scalar quantizer 
being used is that discarding, or zeroing out the k least 
significant bits from the magnitude representation of the 
Index of a quantized coefficient from subband I is equiv- 
alent to scalar quantization of that coefficient with a step- 
size of 2^ A|. Thus. If the compressed bit-stream corre- 
sponding to the codeblock is truncated so that the data 
corresponding to the last k bit-planes is discarded, it is 
possible to reconstruct a more coarsely quantized ver- 
sion of the codeblock. This is known as the embedding 
property. It should k>e noted that if the last k bit-planes 
of the magnitude representation of a quantized coeffi- 
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cient are dropped, for reoonstrucHon at the decoder, the 
reconstruction levels for the quantizer with a step-size 
of 2^ A{ are used. 

[001 8] For the purpose of entropy coding, a bit-plane 
for a codeblodc is said to be significant if any of the pre- s 
vious bitislanes were significant or the bit-plane has at 
least one non-zero bit. The entropy encoder (203) codes 
each bit-plane for the codeblodc In one or niore coding 
passes. For example, the most significant bit-plane Is 
encoded using a single coding pass. The rest of the bit- io 
planes for the codebiock are encoded using three cod- 
ing passes. In JPEG2000, the MQ arithmetic coder is 
used as the entropy coder. The table generation unit 
(204) generates a byte-count table (1 08) for each oode- 
Wock. The m* entry In the table corresponds to the is 
number of bytes needed to Include coding passes 
1 ,2,....m of the codebiock in the bit-stream. 
[0019] The layer formation and ordering decision unit 
(111) determines the number of coding passes to be In- 
cluded in each layer so that the visual quality criteria as 20 
specified by TabieQ (110) are met TableQ (110) stores 
infomnation about the desired visual quality levels. The 
1 «i column entry of the p row. TableQ[I][1 ], specifies the 
resolution at which the Image at visual quality level J will 
be stored. The 2"^ column entry. TableQ[n[2]. specifies 2S 
the viewing condition parameters applicable for visual 
quality level J. As will be described shortly, the viewing 
condition parameters determine the effective quantizer 
step-size to t>e used in each subband and thus, deter- 
mine the visual quality level for the reconstructed image. 30 
The number of layers. L, for each codebiock is equal to 
the number of visual quality levels in TableQ. 
[0020] For a given visual quality level and the corre- 
sponding viewing condition parameters, for each sub- 
band, the layer formation and ordering decision unit 35 
(111) calculates the quantizer step-size that will result In 
Just nof/ceab/e distortion in the reconstructed image. 
This is accomplished by using the two-dimensional Con- 
trast Sensitivity FuncOon (CSF) of the human visual sys- 
tem (HVS). The CSF model described in Jones, "Com- 40 
parative study of wavelet and DCT decomposition with 
equivalent quantization and encoding strategies for 
medical Images". Proc. SPIE Medical Imaging '95, vol. 
2431. pp. 571-682. which is incorporated herein by ref- 
erence, models the sensitivity of the human visual sys- ^5 
tern as a function of the two-dimensional (2-D) spatial 
frequency, and it depends on a numl>er of parameters, 
such as viewing distance, light level, color, image size, 
eccentricity, noise level of the display, and so forth 
These parameters will be referred to as viewing condl- so 
tion parameters. The frequency dependence of the CSF 
Is commonly represented using cycles/degree of visual 
subtense. The CSF can be mapped to other units, such 
as cydes/mm, for a given viewing distance (that is. the 
distance from the observer to the displayed image). ss 
[0021] The 2-D CSF value for subband i is denoted 
by CSF(F,.VC). where VC refers to the viewing condition 
parameters which include the intended viewing dis- 



tance, the ambient light level, the display noise level, 
and the dots per inch for the display. F| represents the 
2-D spatial frequency (In cydes/mm) assodated with 
subband 1. In a preferred embodiment F| is chosen to 
l>e the center of the frequency range nominally associ- 
ated with subband 1. As described in the Jones, if sub- 
band 1 is quantized with a uniform scalar quantizer hav- 
ing a dead-zone, the step-size Q|(VC) that results Injust 
noticeable distortion in the reconstructed image for. 
viewing condition parametere VC is 

Q|(VC)=^ X MTF(F,) X G, X CSF(F,, VC)' 

where MTF(Fi) is the display MTF at frequency F,, C is 
the contrast per code^alue of the display device, and 
Gj is the gain factor that represents the. change in con- 
trast for the reconstructed image for one code-value 
change in a coeffident of subband 1. The gain factor de- 
pends on the level and orientation of the subband, as 
well as the subband synthesis filters. Compared to the 
paper by Jones et al, a foctor of 0.5 is missing from the 
denominator. This is due to the fact that for uniform sca- 
lar quantizer with a dead-zone, the maximum possible 
distortion, E^. is equal to the step-size, as opposed to 
half the step-size for a uniform scalar quantizer in the 
absence of a dead-zone. 

[0022] The viewing condition parameters are pre- 
specified for each visual quality level. Images stored at 
higher visual quality levels are generally viewed under 
more exacting conditions, for example smaller intended 
viewing distances. In some cases, the viewing condition 
parameters may not correspond exactly to the actual 
viewing conditions. For example, suppose that a digitel 
camera captures images at a high enough resolution to 
make 5 by 7 inch prints. Furthermore, suppose that 
three visual quality levels, good, better and best are of- 
fered at that resolution. If the printe are to be viewed at 
a viewing distence of 10 inches, the Intended viewing 
distance for the best quality is specified to t>e 10 inches. 
The viewing distences for the remaining two visual qual- 
ity levels are set to higher values, for example, 1 2 Inches 
for better visual quality and 16 inches for good visual 
quality. All the remaining viewing condition parametere 
are the same for the three quality levels. This results in 
higher quantizer step-sizes for the good and better vis- 
ual quality levels, resulting In suprathreshold quantiza- 
tion artifacts in the reconstructed Images. The visual dis- 
tortion for the image reconstructed at the t>est visual 
quality level is Just noticeable. Similarly the display noise 
levels can be increased beyond the actual value while 
keeping the other viewing condition parametere the 
same, to provide lower visual quality levels. 
[0023] Now consider a codebiock from subband I 
quantized with a step-size A{. Suppose that m coding 
passes for the codebtock are induded in the com- 
pressed bit-stream, and the codebiock is reconstructed 
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from the m coding passes. Let E^ax ^ *h© maximum 
absolute difference between the original codeblock and 
the reconstructed codeblock. The codeblock can be 
thought of as being quantized by a uniform scalar quan- 
tizer with a dead-zone having a step-size of E^^. In this s 
case the effective step-size is E,nax- Assume that for the 
codeblock from subband I. Pq.^) coding passes have 
been included in coding layers 1 ,2,....{1-1), and it is nec- 
essary to determine the number of coding passes to be 
Included In layer j so that the visual quality level j Is at- io 
tained (or exceeded) by including the first j layers in the 
bit-stream. The quantizer step-size Q|(VCj) that results 
in Just noticeable distortion in the reconstructed image 
is calculated. Here VC^ refers to the viewing condition 
parameters at visual quality level J. Then, for coding is 
passes Pq.i).Pq.i)+1. ...and so forth, the effective quan-; 
tizer step-size is calculated. Suppose that at pass m. the 
effective quantizer step-size becomes less than or equal 
to Q,(VCj), then Pj Is set to m. This means that (m -Pq-i)) 
additional coding passes are required In the j* layer of 20 
that codeblock. This process is repeated for each code- 
block from each subband. and for each visual quality 
level. 

[0024] It Is possible that some layers in a codeblock 
may contain no data. For example, consider that an im- 25 
age is to be stored at visual quality level 3, and let the 
corresponding resolution be r. Now, suppose for a code- 
block from subtxand i, the number of coding passes to 
be included in layer 3 is to be determined. As described 
previously, if 1 is greater than Sr. subband I is not neces- 30 
sary to reconstruct the image at resolution r. So no data 
is included in layer 3 of codebtocks belonging to sub- 
t>and i. 

[0025] A flow chart for the layer formation and order- 
ing decision unit (111 ) is shown In FIG. 6. Let the number 35 
of layers for the codeblock be L. and the total number 
of coding passes for the codeblock be P. The inputs to 
the layer formation and ordering decision unit are: 1) 
The visual quality table, TableQ (110), witii L entries. 2) 
ttie original codeblock (105). 3) ttie compressed bit- ^ 
stream corresponding to that codeblock (107). and 4) 
the byte-count table, denoted by TableB (108) for the 
codeblock. The layer fomiation and ordering unit (111) 
generates a table, denoted by TableL (113). that has L 
rows and 2 columns. The 1^ entry from row j denotes ^5 
tiie number of coding passes belonging to layer J. and 
ttie 2^^ entry indicates ttie number of bytes in layer] of 
that codeblock. 

[0026] The Initializer unit (501 ) Initializes J, m, and the 
number of cumulative passes, CP. to zero. It also Initial- so 
Izes the maximum absolute error, E„^. to the maximum 
absolute value of the Indices of quantized coefficients 
for tiiat codeblock. In step (502), j is Incremented by 1 . 
Then, the comparison unit (503) compares] against the 
number of layers, L. If J is greater than 1^ all the layers ss 
have been formed, and the process is stopped after writ- 
ing out TableL (113). Ottienvise. the process is contin- 
ued, in step (504), the resolution r Is set to TabieQQKI]. 



A second comparison unit (505) compares the index of 
tiie subband to which the codebk>ck belongs, i, witti Sr. 
If i is greater than 3r, subband 1 is not needed to recon- 
struct the image at resolution r. The flow-control Is 
passed to step (510) and the row of TableL Is set to 
0. This signifies that no coding passes from the code- 
block are included in layer], if 1 is less than or equal to 
3r, another comparison unit (506) compares E^ax 
ttie quantizer step-size, Q{CVC,), that produces yi/sf no- 
ticeable distortion for viewing condition parameters cor- 
responding to visual quality level j. If E^ax less than 
or equal to Q|(VCj), the flow-control passes to step 
(510), and all coding passes up to and including m are 
included in layer J. Otherwise, m is compared against 
ttie total number of passes, P (507). If m is greater than 
or equal to P, the flow-control skips to step (510). Oth- 
erwise, m is incremented by 1 (508). Then, the code- 
block is reconstructed by using compressed data corre- 
sponding to the first m coding passes, and the maximum 
absolute difference, Emax* between ttie original code- 
block and the reconstructed codeblock is found (509). 
After this, ttie flow-control returns to step (506). In step 
(510), TableL[D[1] Is set to (m - CP) and TableLQlP] is 
set to (TableB[m] - TableB[CP]). Also, the number of cu- 
mulative passes is set to m. Then, the flow-control re- 
turns to step (502). Thus, steps 506 through 509 have 
the effect of Identifying a minimal set of passes and their 
corresponding compressed bit-streams that are neces- 
sary to satisfy each expected visual quality level provid- 
ed in ttie visual quality table. TableQ (110). 
[0027] It should be noted that the step-size used to 
quantize the codeblock should be sufficiently small so 
that when ail the coding passes for the codeblock are 
included In the bit-stream, the maximum quality level 
specified in the visual quality table, TableQ (110). can 
be achieved or exceeded. 

[0028] Another embodiment of the layer formation 
and ordering decision unit is shown In FIG. 6, where an 
additional constraint is placed on the formation of the 
layers. The constraint is that the layer boundaries for a 
block must coincide with the bit-plane boundaries. As 
discussed previously, let the magnitudes of the indtees 
of quantized codeblock coefficients, quantized with a 
step-size of A. be represented by a fixed precision of A 
bits. Let the bits be Indexed as 1 .....A. with index 1 rep- 
resenting the MSB. Now suppose that the k least signif- 
icant bit-planes of the codeblock are discarded. Then, 
tiie effective quantizer step-size for tiie codeblock is (2*^ 
A), instead of calculating the maximum absolute enror. 
E^nax, between the original codeblock and ttie recon- 
structed codeblock as done previously. E„^ Is set equal 
to 2"^ A. 

[0029] In the alternate emt>odiment. the initializer unit 
(601 ) also initializes k to 0. Steps 602 - 606 are identical 
to steps 502 - 506. In step (607). k is compared with A. 
the total number of bit-planes for the codeblock. If k is 
greater than or equal to A. the flow-control passes to 
step (610). Othenvise, in step (608), k is incremented 
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by 1 . and m Is updated so that m represents the number 
of coding passes needed to represent the first k bit- 
planes. In step (609). the effective step-size corre- 
sponding to retaining only the first k bit-pianes. (aC^-*) 
A), is calculated, and E^ax set to this value. Step 610 s 
Is identical to steps 510. Thus, steps 606 tiirough 609 
have the effect of Identifying a minimal set of passes 
and their corresponding compressed bit-streams that 
are necessary to satisfy each expected visual quality 
level provided in ttie visual quality table, TableQ (110). *o 
[0030] Another embodiment of the invention is shown 
in FIG. 7. A compressed bit-stream (701) produced by 
a JPEG2000 encoder is passed tiirough a JPEG2000 
bit-stream parser (718) to produce compressed bit- 
stream corresponding to each codeblock (719). The bit- is 
stream parser also extracts Information ak>out quantizer 
step-sizes (709). Each compressed codeblock bit- 
stream is passed through an entropy decoder (702) to 
produce quantized subband coefficients (703). Steps 
704 - 717 are exactly identical to the con-espondlng 
steps 104 - 117. It should be noted ttiat If the base quan- 
tizer step-sizes used to produce the original JPEG2000 
bit-stream are coarse. It may not be possible to achieve 
ail the visual quality levels from the visual quality table 
(710). 

[0031 ] The Invention can be incorporated into a digital 
still camera 300 shown in FIG. 8. The digital still camera 
300 produces digital images that are stored on a remov- 
able memory card 330. The digital still camera 300 In- 
cludes a zoom lens 312 having zoom and focus motor 
drives 310 and an adjustable aperture and shutter (not 
shown). The user composes tiie image using an optical 
viewfinder 311 and a zoom lens control switch 372 
(shown in FIG. 9). The zoom lens 312 focuses light from 
a scene (not shown) on an image sensor 314, for exam- 
pie, a single-chip color CCD Image sensor, using the 
well-known Bayer color filter pattem. The image sensor 
314 Is controlled by clock drivers 306. The zoom and 
focus motors 310 and the dock drivers 306 are control- 
led by control signals supplied by a control processor 
and timing generator circuit 304. Vymen the user de- 
presses a shutter button 374 to take a picture, the con- 
trol processor and timing generator 304 recedes inputs 
from autofbcus and autoexposure detectors 308 and 
controls a flash 302. The analog output signal from the 
Image sensor 314 Is amplified and converted to digital 
data by an analog signal processing (ASP) and analog- 
to-digital (A/D) converter circuit 316. The digital data is 
stored in a DRAI^ buffer memory 318 and subsequentiy 
processed by a processor 320 controlled by the 
firmware stored in a firmware memory 328, which can 
be flash EPROM memory. 

[0032] The processed digital Image file Is provided to 
a memory card interface 324 which stores ttie digital Im- 
age file on ttie removable memory card 330. The proc- 
essor 320 performs color Interpolation followed by color 
and tone correction, in order to produce rendered sRGB 
Image data. The rendered sRGB image data is then 



compressed using the methods descdl^ed eariier and 
stored as a JPEG 2000 image file on tiie removable 
memory card 330. The processor 320 also creates a 
"screennali" size image that is stored In RAM memory 
326 and supplied to ttie color LCD Image display 332, 
which displays the captured Image for ttie user to review. 
[0033] The graphical user Interface displayed on the 
color LCD image display 332 Is controlled by user Inter- 
nee switches 380, 381 , 382, 383. 384, and 386 shown 
in FIG. 9. After a series of Images has been taken with 
ttie switch 386 In ttie capture position, the switch 386 
can be moved to the review position so that the captured 
pictures can be reviewed on the color LCD image dis- 
play 332. 

[0034] Suppose that in our eariier example, the digital 
still camera 300 has 2 visual quality levels (for example, 
good and best) at the 192 by 128 (thumbnail) resolution 
and the 384 by 256 (screennali) resolution. Further- 
more, the digital still camera 300 has ttiree visual quality 
levels (good, better, and best) each at the 768 by 512 
(emali) resolution and 1536 by 1024 (print) resolution. 
Then, the compressed Image generated by a 
JPEG2000 encoder according to the preferred embod- 
iment has 1 0 quality layers. The JPEG2000 decoder can 
decode just the first quality layer to display a "good* 1 92 
by 128 thumbnail Image. Similariy. ttie JPEG2000 de- 
coder can decode the first 3 quality layers to obtain a 
"besT 384 by 256 image and so on. 
[0035] After a series of images have been taken and 
stored on the removable memory card 330. the remov- 
able memory card 330 can be Inserted Into a memory 
card reader (not shown) in the user's host computer 340. 
Altematively. an interface cable 342 can be used to con- 
nect between the host Interface 322 In the digital still 
camera 300 and the corresponding camera internee In 
the host computer 340. The Interfece cable 342 may 
conform to, for example, the welMcnow universal serial 
bus (USB) interface specification. 
[0036] Initially, the user may wish to download "better" 
quality 768 by 512 resolution images. In that case, only 
the first 6 quality layers are transferred from the digital 
still camera 300 to tiie host computer 340. This may be 
sufficient resolution for emailing ttie image. The inven- 
tion reduces the download time by only transferring the 
quality layers necessary to obtain a user^pedfied qual- 
ity and resolutton. The user may decide that he would 
like to print the image at the highest possible quality. In 
that case, the remaining 4 quality layers are transferred 
from ttie digital still camera 300 to the host computer 
340. 



Clalnts 

1 . A method for producing a compressed digital image 
from an input digital image, wherein the com- 
pressed digital image is organized Into layers hav- 
ing information tiiat can be extracted in accordance 
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with different desired viewing conditions and reso- 
lutions, comprising the steps of: 

(a) decomposing the input digital image to pro- 
duce a plurality of subbands. each subband 
having a plurality of subband coefficients; 

(b) quantizing the plurality of subband coeffi- 
cients of each subband of the decomposed in- 
put digital Image to produce a quantized output 
value for each subband coefficient of each suk>- 
band; 

(c) forming at least one bit-plane from the quan- 
tized output values of subband coefficients of 
each subband; 

(d) entropy encoding each bit-plane of each 
subband in at least one pass to produce a com- 
pressed bit-stream corresponding to each 
pass, wherein each subband is entropy encod- 
ed independently of the other subbands; 

(e) providing a visual quality table that specifies 
a number of expected visual quality levels by 
providing selectable viewing conditions and 
resolutions for each expected visual quality lev- 
el; 

(f) identifying a minimal set of passes and their 
corresponding compressed bit-streams that 
are necessary to satisfy the expected visual 
quality levels provided In the visual quality ta- 
ble, whereby a user can select different desired 
viewing conditions and resolutions for each 
compressed image; and 

(g) ordering the compressed bit-streams corre- 
sponding to passes Into layers from the lowest 
expected visual quality level to the highest ex- 
pected visual quality level specified in the visual 
quality table to produce a compressed digital 
image, wherein each layer includes the passes 
and their corresponding compressed bit- 
streams from ttie identified minimal set corre- 
sponding to the expected visual quality level 
tttat have not been included In any lower visual 
quality layers. 

2. The method according to claim 1 further including 
the step of producing a byte-count table that con- 
tains the number of bytes for the compressed bit- 
stream corresponding to each pass of each bit- 
plane of each subband, and wherein the ordering 
step includes using such byte-count table to identify 
ttie number of bytes sufficient to satisfy each ex- 
pected visual quality level. 

3. The method according to claim 1 wherein ttie iden- 
tifying step is performed using a model of the con- 
trast sensitivity function for ttie human visual sys- 
tem. 

4. A computer program product for causing a compu- 



ter to perform the method of daim 1 . 

5. A method for producing a compressed digital image 
from an input digital Image, wherein the com- 
5 pressed digital image is organized Into layers hav- 
ing infonnation ttiat can be extracted In accordance 
with different desired viewing conditions and reso- 
lutions, comprising tiie steps of: 

10 (a) decomposing the input digital Image to pro- 

duce a plurality of subbands, each subband 
having a plurality of subband coefficients; 

(b) quantizing the plurality of subband coeffi- 
cients of each subband of the decomposed in- 

15 put digital image to produce a quantized output 

value for each subband coefficient of each suk>- 
band; 

(c) partitioning each subband into a pluraii^ of 
codeblocks; 

20 (d) forming at least one bit-plane from the quan- 

tized output values of subband coefficients of 
each codeblock of each subband; 

(e) entropy encoding each bit-plane of each 
codeblock for each subband In at least one 

25 pass to produce a compressed blt-stresun cor- 

responding to each pass, wherein each code- 
block is entropy encoded irulependently of the 
other codeblocks; 

(f) providing a visual quality table that specifies 
30 a number of expected visual quality levels by 

providing selectable viewing conditions and 
resolutions for each expected visual quality lev- 
el; 

(g) identifying a minimal set of passes and their 
35 corresponding compressed bit-streams that 

are necessary to satisfy ttie expected visual 
quality levels provided in the visual quality ta- 
ble, whereby a user can select different desired 
viewing conditions and resolutions for each 
40 compressed image; and 

(h) ordering the compressed bit-streams corre- 
sponding to passes from the lowest expected 
visual quality level to the highest expected vis- 
ual quality level specified in ttie visual quality 

45 table to produce a compressed digital image. 

wherein each layer includes the passes and 
their corresponding compressed bit-streams 
from the Identified minimal set corresponding 
to the expected visual quality level that have not 

50 been included In any lower visual quality layere. 

6. The method according to daim 5 further including 
the step of producing a byte-count table that con- 
tains the number of bytes for the compressed bit- 
55 stream corresponding to each pass of each bit- 
plane of each subband, and wherein the ordering 
step indudes using such byte-count table to identify 
the number of bytes sufficient to satisfy each ex- 
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pected visual quality level. 

7. The method according to claim 5 wherein the Iden- 
tifying step Is performed using a model of the con- 
trast sensitivity function for the human visual sys- s 
tem. 

8. A computer program product for causing a compu- 
ter to perform the method of daim 5. 

10 

9. A digital camera for capturing an Image of a scene 
and for producing a compressed digital image from 
an Input digital Image, wherein the compressed dig- 
ital image is organized into layers having informa- 
tion that can be extracted in accordance with differ- is 
ent desired viewing conditions and resolutions, 
comprising; 

(a) image sensor means for capturing an image 

of the scene and producing a digital Image rep- 20 
resentative of the scene; 

(b) means for processing the digital image to 
provide a processed digital Image; 

(c) decomposing the processed digital Image to 
produce a plurality of subbands, each subband 25 
having a plurality of subband coefHcients; 

(d) quanti2dng the plurality of subt>and coefR- 
cients of each subband of the decomposed dig- 
ital image to produce a quantized output value 

for each subband coefficient of each subt>and: 30 

(e) partitioning each subband into a plurality of 
codebioclcs: 

(f) forming at least one bit-plane from the quan- 
tized output values of subband coefficients of 
each codeblodc of each subband; ^5 

(g) entropy encoding each bit-plane of each 
codeblocic for each subband In at least one 
pass to produce a compressed bit-stream cor- 
responding to each pass, wherein each code- 
bloclc is entropy encoded independently of the ^ 
other codebioclcs; 

(h) providing a visual quality table that specifies 
a number of expected visual quality levels by 
providing selectable viewing conditions and 
resolutions for each expected visual quality lev- 4S 
el; 

(i) identifying a minimal set of passes and their 
corresponding compressed bit-streams that 
are necessary to satisfy the expected visual 
quality levels provided In the visual quality ta- so 
ble. whereby a user can select different desired 
viewing conditions and resolutions for each 
compressed image; 

0) ordering the compressed bit-sbBams conre- 
sponding to passes from the lowest expected ss 
visual quality level to the highest expected vis- 
ual quality level specified in the visual quality 
table to produce a compressed digital image, 



wherein each layer includes the passes and 
their corresponding compressed bit-streams 
from the identified minimal set corresponding 
to the expected visual quality level that have not 
been included in any tower visual quality layers; 
and. 

(k) storage means for storing the compressed 
digital Image. 
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